{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Dataset Descriptions\n",
    "This notebook contains the descriptions of all the datasets used during the tutorials found within this [Learning Pandas repository](https://github.com/tdpetrou/Learn-Pandas).\n",
    "\n",
    "### Datasets\n",
    "* [Employee](#Employee-Data)\n",
    "* [Stack Overflow](#Stack-Overflow-Data)\n",
    "* [Food Inspections](#Food-Inspections-Data)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Helper Function\n",
    "The below **`create_description_table`** function can help create datasets descriptions for any DataFrame. You must first import each DataFrame in as normal and then pass it to the function. You must also pass it a list of the **`descriptions`** as strings."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def create_description_table(df, descriptions, round_num=2):\n",
    "    df_desc = df.dtypes.to_frame(name='Data Type')\n",
    "    df_desc['Description'] = descriptions\n",
    "    df_desc['Missing Values'] = df.isnull().sum()\n",
    "    df_desc['Mean'] = df.select_dtypes('number').mean().round(round_num)\n",
    "    df_desc['Most Common'] = df.apply(lambda x: x.value_counts().index[0])\n",
    "    df_desc['Most Common Ct'] = df.apply(lambda x: x.value_counts().iloc[0])\n",
    "    df_desc['Unique Values'] = df.nunique()\n",
    "    return df_desc"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Employee Data\n",
    "\n",
    "### Brief Overview\n",
    "The city of Houston provides information on all its employees to the public. This is a random sample of 2,000 employees with a selection of the more interesting columns. For more on [open Houston data visit their website](http://data.houstontx.gov/). Data was pulled in December, 2016."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import pandas as pd"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>POSITION_TITLE</th>\n",
       "      <th>DEPARTMENT</th>\n",
       "      <th>BASE_SALARY</th>\n",
       "      <th>RACE</th>\n",
       "      <th>EMPLOYMENT_TYPE</th>\n",
       "      <th>GENDER</th>\n",
       "      <th>HIRE_DATE</th>\n",
       "      <th>JOB_DATE</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>ASSISTANT DIRECTOR (EX LVL)</td>\n",
       "      <td>Municipal Courts Department</td>\n",
       "      <td>121862.0</td>\n",
       "      <td>Hispanic/Latino</td>\n",
       "      <td>Full Time</td>\n",
       "      <td>Female</td>\n",
       "      <td>2006-06-12</td>\n",
       "      <td>2012-10-13</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>LIBRARY ASSISTANT</td>\n",
       "      <td>Library</td>\n",
       "      <td>26125.0</td>\n",
       "      <td>Hispanic/Latino</td>\n",
       "      <td>Full Time</td>\n",
       "      <td>Female</td>\n",
       "      <td>2000-07-19</td>\n",
       "      <td>2010-09-18</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>POLICE OFFICER</td>\n",
       "      <td>Houston Police Department-HPD</td>\n",
       "      <td>45279.0</td>\n",
       "      <td>White</td>\n",
       "      <td>Full Time</td>\n",
       "      <td>Male</td>\n",
       "      <td>2015-02-03</td>\n",
       "      <td>2015-02-03</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>ENGINEER/OPERATOR</td>\n",
       "      <td>Houston Fire Department (HFD)</td>\n",
       "      <td>63166.0</td>\n",
       "      <td>White</td>\n",
       "      <td>Full Time</td>\n",
       "      <td>Male</td>\n",
       "      <td>1982-02-08</td>\n",
       "      <td>1991-05-25</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>ELECTRICIAN</td>\n",
       "      <td>General Services Department</td>\n",
       "      <td>56347.0</td>\n",
       "      <td>White</td>\n",
       "      <td>Full Time</td>\n",
       "      <td>Male</td>\n",
       "      <td>1989-06-19</td>\n",
       "      <td>1994-10-22</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                POSITION_TITLE                     DEPARTMENT  BASE_SALARY  \\\n",
       "0  ASSISTANT DIRECTOR (EX LVL)    Municipal Courts Department     121862.0   \n",
       "1            LIBRARY ASSISTANT                        Library      26125.0   \n",
       "2               POLICE OFFICER  Houston Police Department-HPD      45279.0   \n",
       "3            ENGINEER/OPERATOR  Houston Fire Department (HFD)      63166.0   \n",
       "4                  ELECTRICIAN    General Services Department      56347.0   \n",
       "\n",
       "              RACE EMPLOYMENT_TYPE  GENDER  HIRE_DATE   JOB_DATE  \n",
       "0  Hispanic/Latino       Full Time  Female 2006-06-12 2012-10-13  \n",
       "1  Hispanic/Latino       Full Time  Female 2000-07-19 2010-09-18  \n",
       "2            White       Full Time    Male 2015-02-03 2015-02-03  \n",
       "3            White       Full Time    Male 1982-02-08 1991-05-25  \n",
       "4            White       Full Time    Male 1989-06-19 1994-10-22  "
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "employee = pd.read_csv('../../data/employee.csv', parse_dates=['HIRE_DATE', 'JOB_DATE'])\n",
    "employee.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(2000, 8)"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "employee.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "descriptions = ['Position', 'Department', 'Base salary', 'Race', \n",
    "                'Full time/Part time/Temporary, etc...', 'Gender', \n",
    "                'Date hired', 'Date current job began']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Data Type</th>\n",
       "      <th>Description</th>\n",
       "      <th>Missing Values</th>\n",
       "      <th>Mean</th>\n",
       "      <th>Most Common</th>\n",
       "      <th>Most Common Ct</th>\n",
       "      <th>Unique Values</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>POSITION_TITLE</th>\n",
       "      <td>object</td>\n",
       "      <td>Position</td>\n",
       "      <td>0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>SENIOR POLICE OFFICER</td>\n",
       "      <td>220</td>\n",
       "      <td>330</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>DEPARTMENT</th>\n",
       "      <td>object</td>\n",
       "      <td>Department</td>\n",
       "      <td>0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Houston Police Department-HPD</td>\n",
       "      <td>638</td>\n",
       "      <td>24</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>BASE_SALARY</th>\n",
       "      <td>float64</td>\n",
       "      <td>Base salary</td>\n",
       "      <td>114</td>\n",
       "      <td>55767.93</td>\n",
       "      <td>66614</td>\n",
       "      <td>157</td>\n",
       "      <td>791</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>RACE</th>\n",
       "      <td>object</td>\n",
       "      <td>Race</td>\n",
       "      <td>35</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Black or African American</td>\n",
       "      <td>700</td>\n",
       "      <td>6</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>EMPLOYMENT_TYPE</th>\n",
       "      <td>object</td>\n",
       "      <td>Full time/Part time/Temporary, etc...</td>\n",
       "      <td>0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Full Time</td>\n",
       "      <td>1954</td>\n",
       "      <td>5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>GENDER</th>\n",
       "      <td>object</td>\n",
       "      <td>Gender</td>\n",
       "      <td>0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Male</td>\n",
       "      <td>1397</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>HIRE_DATE</th>\n",
       "      <td>datetime64[ns]</td>\n",
       "      <td>Date hired</td>\n",
       "      <td>0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>2016-03-28 00:00:00</td>\n",
       "      <td>11</td>\n",
       "      <td>999</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>JOB_DATE</th>\n",
       "      <td>datetime64[ns]</td>\n",
       "      <td>Date current job began</td>\n",
       "      <td>3</td>\n",
       "      <td>NaN</td>\n",
       "      <td>2002-01-05 00:00:00</td>\n",
       "      <td>34</td>\n",
       "      <td>947</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                      Data Type                            Description  \\\n",
       "POSITION_TITLE           object                               Position   \n",
       "DEPARTMENT               object                             Department   \n",
       "BASE_SALARY             float64                            Base salary   \n",
       "RACE                     object                                   Race   \n",
       "EMPLOYMENT_TYPE          object  Full time/Part time/Temporary, etc...   \n",
       "GENDER                   object                                 Gender   \n",
       "HIRE_DATE        datetime64[ns]                             Date hired   \n",
       "JOB_DATE         datetime64[ns]                 Date current job began   \n",
       "\n",
       "                 Missing Values      Mean                    Most Common  \\\n",
       "POSITION_TITLE                0       NaN          SENIOR POLICE OFFICER   \n",
       "DEPARTMENT                    0       NaN  Houston Police Department-HPD   \n",
       "BASE_SALARY                 114  55767.93                          66614   \n",
       "RACE                         35       NaN      Black or African American   \n",
       "EMPLOYMENT_TYPE               0       NaN                      Full Time   \n",
       "GENDER                        0       NaN                           Male   \n",
       "HIRE_DATE                     0       NaN            2016-03-28 00:00:00   \n",
       "JOB_DATE                      3       NaN            2002-01-05 00:00:00   \n",
       "\n",
       "                 Most Common Ct  Unique Values  \n",
       "POSITION_TITLE              220            330  \n",
       "DEPARTMENT                  638             24  \n",
       "BASE_SALARY                 157            791  \n",
       "RACE                        700              6  \n",
       "EMPLOYMENT_TYPE            1954              5  \n",
       "GENDER                     1397              2  \n",
       "HIRE_DATE                    11            999  \n",
       "JOB_DATE                     34            947  "
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "create_description_table(employee, descriptions)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Stack Overflow Data\n",
    "This data was gathered from the [Stack Exchange data explorer](https://data.stackexchange.com/), an excellent tool to get almost any data you want from any of the Stack Exchange sites.\n",
    "\n",
    "This particular dataset was collected December 7, 2017 with [this query](http://data.stackexchange.com/stackoverflow/query/768430/get-all-questions-and-answerers-from-tag). You'll have to run the query twice to get all the data because the query exceeds 50,000, the maximum allowable number of rows. Switch the inequality on the **`creationdate`** in the `where` clause to do so."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>creationdate</th>\n",
       "      <th>score</th>\n",
       "      <th>viewcount</th>\n",
       "      <th>title</th>\n",
       "      <th>answercount</th>\n",
       "      <th>commentcount</th>\n",
       "      <th>favoritecount</th>\n",
       "      <th>quest_name</th>\n",
       "      <th>quest_rep</th>\n",
       "      <th>ans_name</th>\n",
       "      <th>ans_rep</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>5486226</td>\n",
       "      <td>2011-03-30 12:26:50</td>\n",
       "      <td>4</td>\n",
       "      <td>2113</td>\n",
       "      <td>Rolling median in python</td>\n",
       "      <td>3</td>\n",
       "      <td>4</td>\n",
       "      <td>1.0</td>\n",
       "      <td>yueerhu</td>\n",
       "      <td>125.0</td>\n",
       "      <td>Mike Pennington</td>\n",
       "      <td>26995.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>5515021</td>\n",
       "      <td>2011-04-01 14:50:44</td>\n",
       "      <td>8</td>\n",
       "      <td>7015</td>\n",
       "      <td>Compute a compounded return series in Python</td>\n",
       "      <td>3</td>\n",
       "      <td>6</td>\n",
       "      <td>7.0</td>\n",
       "      <td>Jason Strimpel</td>\n",
       "      <td>3301.0</td>\n",
       "      <td>Mike Pennington</td>\n",
       "      <td>26995.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>5558607</td>\n",
       "      <td>2011-04-05 21:13:50</td>\n",
       "      <td>2</td>\n",
       "      <td>7392</td>\n",
       "      <td>Sort a pandas DataMatrix in ascending order</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>Jason Strimpel</td>\n",
       "      <td>3301.0</td>\n",
       "      <td>Wes McKinney</td>\n",
       "      <td>43310.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>6467832</td>\n",
       "      <td>2011-06-24 12:31:45</td>\n",
       "      <td>9</td>\n",
       "      <td>13056</td>\n",
       "      <td>How to get the correlation between two timeser...</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>7.0</td>\n",
       "      <td>user814005</td>\n",
       "      <td>117.0</td>\n",
       "      <td>Wes McKinney</td>\n",
       "      <td>43310.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>7577546</td>\n",
       "      <td>2011-09-28 01:58:38</td>\n",
       "      <td>9</td>\n",
       "      <td>2488</td>\n",
       "      <td>Using pandas, how do I subsample a large DataF...</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>5.0</td>\n",
       "      <td>Uri Laserson</td>\n",
       "      <td>958.0</td>\n",
       "      <td>HYRY</td>\n",
       "      <td>54137.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "        id         creationdate  score  viewcount  \\\n",
       "0  5486226  2011-03-30 12:26:50      4       2113   \n",
       "1  5515021  2011-04-01 14:50:44      8       7015   \n",
       "2  5558607  2011-04-05 21:13:50      2       7392   \n",
       "3  6467832  2011-06-24 12:31:45      9      13056   \n",
       "4  7577546  2011-09-28 01:58:38      9       2488   \n",
       "\n",
       "                                               title  answercount  \\\n",
       "0                           Rolling median in python            3   \n",
       "1       Compute a compounded return series in Python            3   \n",
       "2        Sort a pandas DataMatrix in ascending order            2   \n",
       "3  How to get the correlation between two timeser...            1   \n",
       "4  Using pandas, how do I subsample a large DataF...            1   \n",
       "\n",
       "   commentcount  favoritecount      quest_name  quest_rep         ans_name  \\\n",
       "0             4            1.0         yueerhu      125.0  Mike Pennington   \n",
       "1             6            7.0  Jason Strimpel     3301.0  Mike Pennington   \n",
       "2             0            1.0  Jason Strimpel     3301.0     Wes McKinney   \n",
       "3             0            7.0      user814005      117.0     Wes McKinney   \n",
       "4             0            5.0    Uri Laserson      958.0             HYRY   \n",
       "\n",
       "   ans_rep  \n",
       "0  26995.0  \n",
       "1  26995.0  \n",
       "2  43310.0  \n",
       "3  43310.0  \n",
       "4  54137.0  "
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "so = pd.read_csv('../../data/stackoverflow_qa.csv')\n",
    "so.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(56398, 12)"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "so.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "descriptions = ['Question ID', 'Creation date', '# of question upvotes', 'View count',\n",
    "                'Question Title', 'Number of Answers', 'Number of comments for Question',\n",
    "                'Number of favorites for Question', 'User name of question author',\n",
    "                'Reputation of question author', 'User name of selected answer author',\n",
    "                'Reputation of selected answer author']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Data Type</th>\n",
       "      <th>Description</th>\n",
       "      <th>Missing Values</th>\n",
       "      <th>Mean</th>\n",
       "      <th>Most Common</th>\n",
       "      <th>Most Common Ct</th>\n",
       "      <th>Unique Values</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>id</th>\n",
       "      <td>int64</td>\n",
       "      <td>Question ID</td>\n",
       "      <td>0</td>\n",
       "      <td>36953124.64</td>\n",
       "      <td>40190625</td>\n",
       "      <td>1</td>\n",
       "      <td>56398</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>creationdate</th>\n",
       "      <td>object</td>\n",
       "      <td>Creation date</td>\n",
       "      <td>0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>2017-09-13 11:51:07</td>\n",
       "      <td>2</td>\n",
       "      <td>56378</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>score</th>\n",
       "      <td>int64</td>\n",
       "      <td># of question upvotes</td>\n",
       "      <td>0</td>\n",
       "      <td>1.98</td>\n",
       "      <td>0</td>\n",
       "      <td>18530</td>\n",
       "      <td>159</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>viewcount</th>\n",
       "      <td>int64</td>\n",
       "      <td>View count</td>\n",
       "      <td>0</td>\n",
       "      <td>1416.34</td>\n",
       "      <td>38</td>\n",
       "      <td>550</td>\n",
       "      <td>5978</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>title</th>\n",
       "      <td>object</td>\n",
       "      <td>Question Title</td>\n",
       "      <td>0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>cannot write file with full path in Python</td>\n",
       "      <td>2</td>\n",
       "      <td>56392</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>answercount</th>\n",
       "      <td>int64</td>\n",
       "      <td>Number of Answers</td>\n",
       "      <td>0</td>\n",
       "      <td>1.38</td>\n",
       "      <td>1</td>\n",
       "      <td>31299</td>\n",
       "      <td>19</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>commentcount</th>\n",
       "      <td>int64</td>\n",
       "      <td>Number of comments for Question</td>\n",
       "      <td>0</td>\n",
       "      <td>1.65</td>\n",
       "      <td>0</td>\n",
       "      <td>26321</td>\n",
       "      <td>26</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>favoritecount</th>\n",
       "      <td>float64</td>\n",
       "      <td>Number of favorites for Question</td>\n",
       "      <td>43741</td>\n",
       "      <td>2.33</td>\n",
       "      <td>1</td>\n",
       "      <td>7326</td>\n",
       "      <td>81</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>quest_name</th>\n",
       "      <td>object</td>\n",
       "      <td>User name of question author</td>\n",
       "      <td>299</td>\n",
       "      <td>NaN</td>\n",
       "      <td>user308827</td>\n",
       "      <td>241</td>\n",
       "      <td>19848</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>quest_rep</th>\n",
       "      <td>float64</td>\n",
       "      <td>Reputation of question author</td>\n",
       "      <td>291</td>\n",
       "      <td>1637.93</td>\n",
       "      <td>1</td>\n",
       "      <td>1502</td>\n",
       "      <td>2668</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>ans_name</th>\n",
       "      <td>object</td>\n",
       "      <td>User name of selected answer author</td>\n",
       "      <td>19119</td>\n",
       "      <td>NaN</td>\n",
       "      <td>jezrael</td>\n",
       "      <td>5436</td>\n",
       "      <td>4907</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>ans_rep</th>\n",
       "      <td>float64</td>\n",
       "      <td>Reputation of selected answer author</td>\n",
       "      <td>19117</td>\n",
       "      <td>74849.53</td>\n",
       "      <td>186894</td>\n",
       "      <td>5436</td>\n",
       "      <td>2648</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "              Data Type                           Description  Missing Values  \\\n",
       "id                int64                           Question ID               0   \n",
       "creationdate     object                         Creation date               0   \n",
       "score             int64                 # of question upvotes               0   \n",
       "viewcount         int64                            View count               0   \n",
       "title            object                        Question Title               0   \n",
       "answercount       int64                     Number of Answers               0   \n",
       "commentcount      int64       Number of comments for Question               0   \n",
       "favoritecount   float64      Number of favorites for Question           43741   \n",
       "quest_name       object          User name of question author             299   \n",
       "quest_rep       float64         Reputation of question author             291   \n",
       "ans_name         object   User name of selected answer author           19119   \n",
       "ans_rep         float64  Reputation of selected answer author           19117   \n",
       "\n",
       "                      Mean                                 Most Common  \\\n",
       "id             36953124.64                                    40190625   \n",
       "creationdate           NaN                         2017-09-13 11:51:07   \n",
       "score                 1.98                                           0   \n",
       "viewcount          1416.34                                          38   \n",
       "title                  NaN  cannot write file with full path in Python   \n",
       "answercount           1.38                                           1   \n",
       "commentcount          1.65                                           0   \n",
       "favoritecount         2.33                                           1   \n",
       "quest_name             NaN                                  user308827   \n",
       "quest_rep          1637.93                                           1   \n",
       "ans_name               NaN                                     jezrael   \n",
       "ans_rep           74849.53                                      186894   \n",
       "\n",
       "               Most Common Ct  Unique Values  \n",
       "id                          1          56398  \n",
       "creationdate                2          56378  \n",
       "score                   18530            159  \n",
       "viewcount                 550           5978  \n",
       "title                       2          56392  \n",
       "answercount             31299             19  \n",
       "commentcount            26321             26  \n",
       "favoritecount            7326             81  \n",
       "quest_name                241          19848  \n",
       "quest_rep                1502           2668  \n",
       "ans_name                 5436           4907  \n",
       "ans_rep                  5436           2648  "
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "create_description_table(so, descriptions)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Food Inspections Data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>DBA Name</th>\n",
       "      <th>Facility Type</th>\n",
       "      <th>Risk</th>\n",
       "      <th>Address</th>\n",
       "      <th>Zip</th>\n",
       "      <th>Inspection Date</th>\n",
       "      <th>Inspection Type</th>\n",
       "      <th>Results</th>\n",
       "      <th>Violations</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>DANY'S TACOS</td>\n",
       "      <td>Restaurant</td>\n",
       "      <td>Risk 1 (High)</td>\n",
       "      <td>2857 S ST LOUIS AVE</td>\n",
       "      <td>60623.0</td>\n",
       "      <td>2017-03-27</td>\n",
       "      <td>License</td>\n",
       "      <td>Fail</td>\n",
       "      <td>16. FOOD PROTECTED DURING STORAGE, PREPARATION...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>BILLY FOOD MARKET INC</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Risk 3 (Low)</td>\n",
       "      <td>3906 W ROOSEVELT RD</td>\n",
       "      <td>60624.0</td>\n",
       "      <td>2017-03-27</td>\n",
       "      <td>License</td>\n",
       "      <td>Not Ready</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>TAQUERIA HACIENDA TAPATIA</td>\n",
       "      <td>Restaurant</td>\n",
       "      <td>Risk 1 (High)</td>\n",
       "      <td>4125 W 26TH ST</td>\n",
       "      <td>60623.0</td>\n",
       "      <td>2017-03-27</td>\n",
       "      <td>License Re-Inspection</td>\n",
       "      <td>Pass</td>\n",
       "      <td>2. FACILITIES TO MAINTAIN PROPER TEMPERATURE -...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>WILD GOOSE BAR &amp; GRILL</td>\n",
       "      <td>Restaurant</td>\n",
       "      <td>Risk 1 (High)</td>\n",
       "      <td>4265 N LINCOLN AVE</td>\n",
       "      <td>60618.0</td>\n",
       "      <td>2017-03-27</td>\n",
       "      <td>Canvass</td>\n",
       "      <td>Fail</td>\n",
       "      <td>16. FOOD PROTECTED DURING STORAGE, PREPARATION...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>PUBLICAN TAVERN K1</td>\n",
       "      <td>Restaurant</td>\n",
       "      <td>Risk 1 (High)</td>\n",
       "      <td>11601 W TOUHY AVE</td>\n",
       "      <td>60666.0</td>\n",
       "      <td>2017-03-27</td>\n",
       "      <td>Canvass</td>\n",
       "      <td>Fail</td>\n",
       "      <td>18. NO EVIDENCE OF RODENT OR INSECT OUTER OPEN...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                    DBA Name Facility Type           Risk  \\\n",
       "0               DANY'S TACOS    Restaurant  Risk 1 (High)   \n",
       "1      BILLY FOOD MARKET INC           NaN   Risk 3 (Low)   \n",
       "2  TAQUERIA HACIENDA TAPATIA    Restaurant  Risk 1 (High)   \n",
       "3     WILD GOOSE BAR & GRILL    Restaurant  Risk 1 (High)   \n",
       "4         PUBLICAN TAVERN K1    Restaurant  Risk 1 (High)   \n",
       "\n",
       "                Address      Zip Inspection Date        Inspection Type  \\\n",
       "0  2857 S ST LOUIS AVE   60623.0      2017-03-27                License   \n",
       "1  3906 W ROOSEVELT RD   60624.0      2017-03-27                License   \n",
       "2       4125 W 26TH ST   60623.0      2017-03-27  License Re-Inspection   \n",
       "3   4265 N LINCOLN AVE   60618.0      2017-03-27                Canvass   \n",
       "4    11601 W TOUHY AVE   60666.0      2017-03-27                Canvass   \n",
       "\n",
       "     Results                                         Violations  \n",
       "0       Fail  16. FOOD PROTECTED DURING STORAGE, PREPARATION...  \n",
       "1  Not Ready                                                NaN  \n",
       "2       Pass  2. FACILITIES TO MAINTAIN PROPER TEMPERATURE -...  \n",
       "3       Fail  16. FOOD PROTECTED DURING STORAGE, PREPARATION...  \n",
       "4       Fail  18. NO EVIDENCE OF RODENT OR INSECT OUTER OPEN...  "
      ]
     },
     "execution_count": 34,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "food_inspections = pd.read_csv('../../data/food_inspections.csv', parse_dates=['Inspection Date'])\n",
    "food_inspections.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(24063, 9)"
      ]
     },
     "execution_count": 35,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "food_inspections.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "descriptions = ['Doing business as Name', 'Restaurant, Grocery store, School, Bakery, etc...',\n",
    "                'High/Medium/Low', 'Address', 'Zip Code', 'Inspection Date',\n",
    "                'Inspection Type', 'Pass/Fail/Out of business, etc...',\n",
    "                'Detailed description of violations']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Data Type</th>\n",
       "      <th>Description</th>\n",
       "      <th>Missing Values</th>\n",
       "      <th>Mean</th>\n",
       "      <th>Most Common</th>\n",
       "      <th>Most Common Ct</th>\n",
       "      <th>Unique Values</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>DBA Name</th>\n",
       "      <td>object</td>\n",
       "      <td>Doing business as Name</td>\n",
       "      <td>0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>SUPER LEON</td>\n",
       "      <td>1</td>\n",
       "      <td>24063</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Facility Type</th>\n",
       "      <td>object</td>\n",
       "      <td>Restaurant, Grocery store, School, Bakery, etc...</td>\n",
       "      <td>3445</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Restaurant</td>\n",
       "      <td>11849</td>\n",
       "      <td>392</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Risk</th>\n",
       "      <td>object</td>\n",
       "      <td>High/Medium/Low</td>\n",
       "      <td>36</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Risk 1 (High)</td>\n",
       "      <td>12802</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Address</th>\n",
       "      <td>object</td>\n",
       "      <td>Address</td>\n",
       "      <td>0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>11601 W TOUHY AVE</td>\n",
       "      <td>155</td>\n",
       "      <td>15320</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Zip</th>\n",
       "      <td>float64</td>\n",
       "      <td>Zip Code</td>\n",
       "      <td>34</td>\n",
       "      <td>60629.1</td>\n",
       "      <td>60647</td>\n",
       "      <td>967</td>\n",
       "      <td>93</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Inspection Date</th>\n",
       "      <td>datetime64[ns]</td>\n",
       "      <td>Inspection Date</td>\n",
       "      <td>0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>2016-10-05 00:00:00</td>\n",
       "      <td>83</td>\n",
       "      <td>1772</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Inspection Type</th>\n",
       "      <td>object</td>\n",
       "      <td>Inspection Type</td>\n",
       "      <td>1</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Canvass</td>\n",
       "      <td>16422</td>\n",
       "      <td>43</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Results</th>\n",
       "      <td>object</td>\n",
       "      <td>Pass/Fail/Out of business, etc...</td>\n",
       "      <td>0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Pass</td>\n",
       "      <td>11358</td>\n",
       "      <td>7</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Violations</th>\n",
       "      <td>object</td>\n",
       "      <td>Detailed description of violations</td>\n",
       "      <td>12342</td>\n",
       "      <td>NaN</td>\n",
       "      <td>18. NO EVIDENCE OF RODENT OR INSECT OUTER OPEN...</td>\n",
       "      <td>12</td>\n",
       "      <td>11679</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                      Data Type  \\\n",
       "DBA Name                 object   \n",
       "Facility Type            object   \n",
       "Risk                     object   \n",
       "Address                  object   \n",
       "Zip                     float64   \n",
       "Inspection Date  datetime64[ns]   \n",
       "Inspection Type          object   \n",
       "Results                  object   \n",
       "Violations               object   \n",
       "\n",
       "                                                       Description  \\\n",
       "DBA Name                                    Doing business as Name   \n",
       "Facility Type    Restaurant, Grocery store, School, Bakery, etc...   \n",
       "Risk                                               High/Medium/Low   \n",
       "Address                                                    Address   \n",
       "Zip                                                       Zip Code   \n",
       "Inspection Date                                    Inspection Date   \n",
       "Inspection Type                                    Inspection Type   \n",
       "Results                          Pass/Fail/Out of business, etc...   \n",
       "Violations                      Detailed description of violations   \n",
       "\n",
       "                 Missing Values     Mean  \\\n",
       "DBA Name                      0      NaN   \n",
       "Facility Type              3445      NaN   \n",
       "Risk                         36      NaN   \n",
       "Address                       0      NaN   \n",
       "Zip                          34  60629.1   \n",
       "Inspection Date               0      NaN   \n",
       "Inspection Type               1      NaN   \n",
       "Results                       0      NaN   \n",
       "Violations                12342      NaN   \n",
       "\n",
       "                                                       Most Common  \\\n",
       "DBA Name                                                SUPER LEON   \n",
       "Facility Type                                           Restaurant   \n",
       "Risk                                                 Risk 1 (High)   \n",
       "Address                                         11601 W TOUHY AVE    \n",
       "Zip                                                          60647   \n",
       "Inspection Date                                2016-10-05 00:00:00   \n",
       "Inspection Type                                            Canvass   \n",
       "Results                                                       Pass   \n",
       "Violations       18. NO EVIDENCE OF RODENT OR INSECT OUTER OPEN...   \n",
       "\n",
       "                 Most Common Ct  Unique Values  \n",
       "DBA Name                      1          24063  \n",
       "Facility Type             11849            392  \n",
       "Risk                      12802              4  \n",
       "Address                     155          15320  \n",
       "Zip                         967             93  \n",
       "Inspection Date              83           1772  \n",
       "Inspection Type           16422             43  \n",
       "Results                   11358              7  \n",
       "Violations                   12          11679  "
      ]
     },
     "execution_count": 37,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "create_description_table(food_inspections, descriptions)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
